The present invention relates to real-time parallel processing using so-called liquid architectures, and, more particularly but not exclusively, to real-time processing and classification of streaming noisy data using adaptive, asynchronous, fault tolerant, robust, and parallel processors.
During the last decade, there has been a growing demand for solutions to the computing problems of Turing-machine (TM)-based computers, which are commonly used for interactive computing. One suggested solution is a partial transition from interactive computing to proactive computing. Proactive computers are needed, inter alia, for providing fast computing of natural signals from the real world, such as sound and image signals. Such fast computing requires the real time processing of massive quantities of asynchronous sources of information. The ability to analyze such signals in real time may allow the implementation of various applications, which are designed for tasks that currently can be done only by humans. In proactive computers, billions of computing devices may be directly connected to the physical world so that I/O devices are no longer needed.
As proactive computers are designed to allow the execution of day-to-day tasks in the physical world, an instrument that constitutes the connection to the real world must be part of the process, so that the computer systems will be exposed to, and linked with, the natural environment. In order to allow such linkages, the proactive computers have to be able to convert real world signals into digital signals. Such conversions are needed for performing various tasks which are based on analysis of real world natural signals, for example, human speech recognition, image processing, textual and image content recognition, such as optical character recognition (OCR) and automatic target recognition (ATR), and objective quality assessment of such natural signals.
Regular computing processes are usually based on TM computers which are configured to compute deterministic input signals. As commonly known, occurrences in the real world are unpredictable and usually do not exhibit deterministic behavior. Execution of tasks which are based on analysis of real world signals have high computational complexity and, thus, analysis of massive quantities of noisy data and complex structures and relationships is needed. As the commonly used TM based computers are not designed to handle such unpredictable input signals, in affective manner, the computing process usually requires high computational power and energy source power.
Gordon Moore's Law predicts exponential growth of the number of transistors per integrated circuit. Such exponential growth is needed in order to increase the computational power of signal chip processor, however as the transistors become smaller and reduce the effective length of the distance in the near-surface region of a silicon substrate between edges of the drain and source regions in the field effect transistor is reduced, and it becomes practically impossible to synchronize the entire chip. The reduced length can be problematic; as such a large number of transistors may be leaky, noisy, and unreliable. Moreover, fabrication cost grows each year as it becomes increasingly difficult to synchronize an entire chip at multiple GHz clock rates and to perform design verification and validation of a design having more than 100 million transistors.
In the light of the above, it seems that TM-based computers have a growth limit and, therefore, may not be the preferred solution for analyzing real world natural signals. An example of a pressing problem that requires analysis of real world signals is speech recognition. Many problems have to be solved in order to provide an efficient generic mechanism for speech recognition. However, most of the problems are caused by the unpredictable nature of the speech signals. For example, one problem is due to the fact that different users have different voices and accents, and, therefore, speech signals that represent the same words or sentences have numerous different and unpredictable structures. In addition, environmental conditions such as noise, channel limitations, and may also have an effect on the performance of the speech recognition.
Another example of pressing problem which is not easily solved by TM-based computers is related to the field of string matching and regular expressions identification. Fast string matching and regular expression detection is necessary for a wide range of applications, such as information retrieval, content inspection, data processing and others. Most of the algorithms available for string matching and regular expression identification are endowed with high computational complexity and, therefore, require many computational sources. A known solution to the problem requires a large amount of memory for storing all the optional strings and hardware architecture, as it is based on the Finite-State-Machine (FSM) model, wherein the memory for each execution of matching operations is sequentially accessed. Such a solution requires, in turn, large memory arrays that constitute a bottleneck that limits throughput, since the access to memory is a time or clock cycle consuming operation. Therefore, it is clear that a solution that allows the performance of string matching yet can save on access to memory, and can substantially improve the performance of the process.
During the last decade, a number of non-TM computational solutions have been adopted to solve the problems of real world signals analysis. A known computational architecture which has been tested is neural network. A neural network is an interconnected assembly of simple nonlinear processing elements, units or nodes, whose functionality is loosely based on the animal brain. The processing ability of the network is stored in the inter-unit connection strengths, or weights, obtained by a process of adaptation to, or learning from, a set of training patterns. Neural nets are used in bioinformatics to map data and make predictions. However, a pure hardware implementation of a neural network utilizing existing technology is not simple. One of the difficulties in creating true physical neural networks lies in the highly complex manner in which a physical neural network must be designed and constructed.
One solution, which has been proposed for solving the difficulties in creating true physical neural networks, is known as a liquid state machine (LSM). An example of an LSM is disclosed in “Computational Models for Generic Cortical Microcircuits” by Wolfgang Maass et al., of the Institute for Theoretical Computer Science, Technische Universitaet Graz, Graz, Austria, published on Jan. 10, 2003. The LSM model of Maass et al. comprises three parts: an input layer, a large randomly connected core which has the intermediate states transformed from input, and an output layer. Given a time series as input, the machine can produce a time series as a reaction to the input. To get the desired reaction, the weights on the links between the core and the output must be adjusted.
U.S. Patent Application No. 2004/0153426, published on Aug. 5, 2004, discloses the implementation of a physical neural network using a liquid state machine in nanotechnology. The physical neural network is based on molecular connections located within a dielectric solvent between presynaptic and postsynaptic electrodes thereof, such that the molecular connections are strengthened or weakened according to an application of an electric field or a frequency thereof to provide physical neural network connections thereof. A supervised learning mechanism is associated with the liquid state machine, whereby connection strengths of the molecular connections are determined by presynaptic and postsynaptic activity respectively associated with the presynaptic and postsynaptic electrodes, wherein the liquid state machine comprises a dynamic fading memory mechanism.
Another type of network, very similar to the LSM, is known as an echo state net (ESN) or an echo state machine (ESM), which allows universal real-time computation without stable state or attractors on continuous input streams. From an engineering point of view, the ESN model seems nearly identical to the LSM model. Both use the dynamics of recurrent neural networks for preprocessing input and train extra mechanisms for obtaining information from the dynamic states of these networks. An ESN based neural network consists of a large fixed recurrent reservoir network from which a desired output is obtained by training suitable output connection weights. Although these systems and methods present optional solutions to the aforementioned computational problem, the solutions are complex and in any event do not teach how the liquid state machine can be efficiently used to solve some of the signal processing problems.
There is thus a widely recognized need for, and it would be highly advantageous to have, a method and a system for processing stochastic noisy natural signals in parallel computing devoid of the above limitations.